In [1]:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import cv2, glob, math
from collections import deque
from moviepy.editor import VideoFileClip
from IPython.display import HTML
%matplotlib inline
Camera Calibration¶
We use the images in the camera_cal folder to compute the matrix and distortion coefficients. These are computed once and can be used later repetedly throughout the pipeline.
Q. In the first place, why does the distortion happen?
With the cheap pinhole cameras, when the 3D pictures get transformed into the 2D space, there might be distortions introduced. This issue occurs when mapping the 3D points to 2D.
Q. How do we correct them?
These distortions remain constant and can be remapped easily by using some calibration methodologies. Below, we will be using one of Opencv's methods to remove the distortions. We repeatedly use this matrix and the coefficients that will help us undistort the frames in our video.
In [2]:
objp = np.zeros((6*9,3), np.float32)
objp[:,:2] = np.mgrid[0:9, 0:6].T.reshape(-1,2)
objpoints = [] # 3d points in real world space
imgpoints = [] # 2d points in image plane.
images = glob.glob('camera_cal/calibration*.jpg')
for idx, fname in enumerate(images):
img = cv2.imread(fname)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, corners = cv2.findChessboardCorners(gray, (9,6), None)
if ret == True:
objpoints.append(objp)
imgpoints.append(corners)
cv2.drawChessboardCorners(img, (9,6), corners, ret)
plt.figure(figsize=(15,9))
plt.imshow(img)
# cv2.imwrite('rubric_images/{}'.format(fname.split('/')[-1]), img)
cv2.waitKey(500)
cv2.destroyAllWindows()
img = cv2.cvtColor(cv2.imread("camera_cal/calibration1.jpg"), cv2.COLOR_BGR2RGB)
img_size = (img.shape[1], img.shape[0])
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, img_size, None, None)
Let's check if the camera calibration has worked by undistorting one of the calibration images by using the pre-computer matrix and coeffiicients¶
In [3]:
def undistort_image(img):
'''
Undistort the image based on the matrix and coefficients calculated during calibration.
:param any image that is to be undistorted
:return undistorted image
'''
dst = cv2.undistort(img, mtx, dist, None, mtx)
return dst
In [4]:
img = cv2.cvtColor(cv2.imread("camera_cal/calibration5.jpg"), cv2.COLOR_BGR2RGB)
dst = undistort_image(img)
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,10))
ax1.imshow(img);
ax1.set_title('Original Image', fontsize=30);
ax2.imshow(dst);
ax2.set_title('Undistorted Image', fontsize=30);
As we can see, the curvish part on the left has been straightened. However, since we calibrated using the above original image, it is imperative that the undistortion would work. Let's try experimenting on one of our images from the dataset we have for the project.¶
In [5]:
img = cv2.cvtColor(cv2.imread("test_images/test1.jpg"), cv2.COLOR_BGR2RGB)
dst = undistort_image(img)
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,10))
ax1.imshow(img);
ax1.set_title('Original Image', fontsize=30);
ax2.imshow(dst);
ax2.set_title('Undistorted Image', fontsize=30);
At once, one might not notice the suttle differences in the above original and distortion corrected image. We can see that the hood has somewhat been suppressed. To visualize one might think a boomerang straightened a bit.
Color Spaces and Gradients¶
Q. Have I used color spaces and gradients both in the pipeline?
No. Only color spaces sufficed the need. Besides, gradients didn't seem to give much of a boost to the pipeline and instead created a lot of noise.
Q. What are the color spaces that I experimented with?
Since, the code was pretty much the same with change to only a single parameter, I checked the results of HLS, LAB, LUV, HSV
Q. Which one did I use or rather which combination did I choose?
S channel from the HLS space, V channel from the HSV space
Below are the results of some of the experiments I conducted with some of the color spaces I mentioned above
HLS¶
In [6]:
def show_color_space_results(channel='HLS'):
'''
Display the various channel outputs of the BGR image/frame.
:param channel of which the plots are desired
:return None
'''
for filepath in glob.glob("test_images/*.jpg"):
if channel == 'LAB':
img = cv2.cvtColor(cv2.imread(filepath), cv2.COLOR_BGR2LAB)
elif channel == 'LUV':
img = cv2.cvtColor(cv2.imread(filepath), cv2.COLOR_BGR2LUV)
elif channel == 'HSV':
img = cv2.cvtColor(cv2.imread(filepath), cv2.COLOR_BGR2HSV)
else:
img = cv2.cvtColor(cv2.imread(filepath), cv2.COLOR_BGR2HLS)
c0 = img[:,:,0]
c1 = img[:,:,1]
c2 = img[:,:,2]
f, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(20,15))
ax1.imshow(c0)
ax1.set_title("{} channel".format(channel[0]))
ax2.imshow(c1)
ax2.set_title("{} channel".format(channel[1]))
ax3.imshow(c2)
ax3.set_title("{} channel".format(channel[2]))
In [7]:
show_color_space_results('HLS')
We can see that the S channel shows sharp results for the lane lines and it seems that the higher pixel thresholds might easily be able to differentiate lane lines from the surrounding.
LUV¶
In [8]:
show_color_space_results('LUV')
The U and V channels seem to completely miss the right lanes. L channel show some good performance in the dark or normal conditions. I used the L channel in my initial pipeline, but it does seem to fail terribly in conditions where brightness is high.
HSV¶
In [9]:
show_color_space_results('HSV')
As I mentioned, the V channel showed a lot of promise. Though similar to L channel from LUV in case of brighter conditions, it performs slightly better and with proper tuning of the thresholds, it works better and hence I chose to go ahead with it
Perspective Transform (Birds Eye view)¶
What is a Perspective Transform?
Simply put, the perspective transform is just a change in the viewing angle. Suppose a person is driving a car on lane. The way the driver looks at the lane he is driving on is different from the person who is on the side seat as well as the back seats. So, each of these cases, certain person might have better view of what lies ahead. For example, the driver might have a better view of curves that are ahead of them. However, if imagine a bird watching over the road from above, it would have an even better view of the road ahead. So, these are ways by which a single road/lane can be viewed from different angles.
Whats he use?
We use the best of worlds to choose what action to take further. (Note: We are not taking action in this part of the project, but will definitely later.) So, by viewing using a birds-eye view i.e. top view, we can estimate that there are curves ahead better than just viewing from the driver's perspective. We will view some images of the drivers and the birds view of the same road.
*Q. How do we use it in our pipeline?
We binarize the image using some thresholds and finally what we get is lane lines from the top view. We then use these lines to decide if the line is straight, or turns left or right. And depending upon that we fit polynomials that will draw straight lines or curves onto the original image. Simply put, these lines will be drawn based on coordinates and hence in the further projects, we can train the car or simulator to stay within these points.
In [10]:
def binarize(b, thresh=(210, 255)):
'''
Binarize the image depending on the thresholds
:param image to be binarized
:param threshold tuple with lower and upper bounds of tolerance
:return binarized image
'''
binary = np.zeros_like(b)
binary[(b >= thresh[0]) & (b <= thresh[1])] = 1
return binary
def combine_thresholds(binary_1, binary_2):
'''
We use the S channel from the HLS and V from HSV. Binarize each of them individually and in this function
combine both of them to form a single combined binary image
:param s channel binarized image
:param v channel binarized image
:return combined binary image of the params
'''
combined_binary = np.zeros_like(binary_1)
combined_binary[(binary_1 == 1) | (binary_2 == 1)] = 1
return combined_binary
def threshold_image(img, s_thresh=(200,240), v_thresh=(230,255)):
'''
Extracts out the respective channels from the original image and delegates the functionality of binarization
and thresholding and returns back the combined binary image
:param orginal RGB image
:param s channel thresholds
:param v channel thresholds
:return combined binared image
'''
s = cv2.cvtColor(img, cv2.COLOR_RGB2HLS)[:,:,2]
v = cv2.cvtColor(img, cv2.COLOR_RGB2HSV)[:,:,2]
s_b = binarize(s, s_thresh)
v_b = binarize(v, v_thresh)
return combine_thresholds(s_b, v_b)
def warp(img, flag=False):
'''
This function lets us get a birds-eye view of image irrespective of its orientation. Most often than not,
in the current pipeline we use this function to convert binarized image into warped (birds eye view) image
:param any image (combined binary image here)
:param if True, returns only the warped image, useful for single image display
:return warped image
'''
src = np.float32([[(180,720), (560,460), (740,460), (1200,720)]])
dest = np.float32([[(100,720), (0,0), (1300,0), (1200,720)]])
M = cv2.getPerspectiveTransform(src, dest)
MInv = cv2.getPerspectiveTransform(dest, src)
warped = cv2.warpPerspective(img, M, img_size, flags=cv2.INTER_LINEAR)
if flag:
return warped
return warped, M, MInv
In [11]:
for filepath in glob.glob("test_images/*.jpg"):
img = cv2.cvtColor(cv2.imread(filepath), cv2.COLOR_BGR2RGB)
warped = warp(img, True)
binary_representation = threshold_image(warped)
f, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(15,9))
ax1.imshow(img);
ax1.set_title("Driver View")
ax2.imshow(warped);
ax2.set_title("Birds Eye view")
ax3.imshow(binary_representation, cmap='gray');
ax3.set_title("Birds Eye view binary")
Fitting the lines¶
This is the first version that uses the sliding window algorithm to detect the lane lines and draw rectangles. Later we will improve the below method where we detect if the previous frame had the required margin detected, in that case, we skip calculating the x, y coordinates and continue depending on the recent averages of the found values.
In [12]:
def find_lane_pixels(binary_warped):
'''
Find the lane pixels which will be tracked to fit the polynomial and draw sliding windows and later
lane polygon.
:param binary warped image
:return the coordinates of the points where the margins are suspected to begin from where the
windows are drawn.
'''
# Take a histogram of the bottom half of the image
histogram = np.sum(binary_warped[binary_warped.shape[0]//2:,:], axis=0)
# Create an output image to draw on and visualize the result
out_img = np.dstack((binary_warped, binary_warped, binary_warped))
# Find the peak of the left and right halves of the histogram
# These will be the starting point for the left and right lines
midpoint = np.int(histogram.shape[0]//2)
leftx_base = np.argmax(histogram[:midpoint])
rightx_base = np.argmax(histogram[midpoint:]) + midpoint
# HYPERPARAMETERS
nwindows = 10
margin = 100
minpix = 20
# Set height of windows - based on nwindows above and image shape
window_height = np.int(binary_warped.shape[0]//nwindows)
# Identify the x and y positions of all nonzero pixels in the image
nonzero = binary_warped.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
# Current positions to be updated later for each window in nwindows
leftx_current = leftx_base
rightx_current = rightx_base
left_lane_inds = []
right_lane_inds = []
for window in range(nwindows):
win_y_low = binary_warped.shape[0] - (window+1)*window_height
win_y_high = binary_warped.shape[0] - window*window_height
win_xleft_low = leftx_current - margin
win_xleft_high = leftx_current + margin
win_xright_low = rightx_current - margin
win_xright_high = rightx_current + margin
cv2.rectangle(out_img,(win_xleft_low,win_y_low),
(win_xleft_high,win_y_high),(0,255,0), 2)
cv2.rectangle(out_img,(win_xright_low,win_y_low),
(win_xright_high,win_y_high),(0,255,0), 2)
good_left_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) &
(nonzerox >= win_xleft_low) & (nonzerox < win_xleft_high)).nonzero()[0]
good_right_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) &
(nonzerox >= win_xright_low) & (nonzerox < win_xright_high)).nonzero()[0]
left_lane_inds.append(good_left_inds)
right_lane_inds.append(good_right_inds)
if len(good_left_inds) > minpix:
leftx_current = np.int(np.mean(nonzerox[good_left_inds]))
if len(good_right_inds) > minpix:
rightx_current = np.int(np.mean(nonzerox[good_right_inds]))
try:
left_lane_inds = np.concatenate(left_lane_inds)
right_lane_inds = np.concatenate(right_lane_inds)
except ValueError:
pass
leftx = nonzerox[left_lane_inds]
lefty = nonzeroy[left_lane_inds]
rightx = nonzerox[right_lane_inds]
righty = nonzeroy[right_lane_inds]
return leftx, lefty, rightx, righty, out_img
def fit_polynomial(binary_warped):
'''
Fit the polynomial to the binary warped image
:param binary warped image on which the lane lines be found and drawn
:return resultant image with left and right margins identified and windows drawn on.
'''
leftx, lefty, rightx, righty, out_img = find_lane_pixels(binary_warped)
left_fit = np.polyfit(lefty, leftx, 2)
right_fit = np.polyfit(righty, rightx, 2)
ploty = np.linspace(0, binary_warped.shape[0]-1, binary_warped.shape[0] )
try:
left_fitx = left_fit[0]*ploty**2 + left_fit[1]*ploty + left_fit[2]
right_fitx = right_fit[0]*ploty**2 + right_fit[1]*ploty + right_fit[2]
except TypeError:
print('The function failed to fit a line!')
left_fitx = 1*ploty**2 + 1*ploty
right_fitx = 1*ploty**2 + 1*ploty
out_img[lefty, leftx] = [255, 0, 0]
out_img[righty, rightx] = [0, 0, 255]
return out_img, left_fitx, right_fitx, ploty
In [13]:
filepaths = ["test_images/test2.jpg","test_images/test3.jpg", "test_images/test4.jpg"]
# filepaths = glob.glob('test_images/*.jpg')
for filepath in filepaths:
img = cv2.cvtColor(cv2.imread(filepath), cv2.COLOR_BGR2RGB)
warped = warp(img, True)
binary_repr = threshold_image(warped)
out_img, left_fitx, right_fitx, ploty = fit_polynomial(binary_repr)
plt.figure(figsize=(10,8))
plt.plot(left_fitx, ploty, color='yellow')
plt.plot(right_fitx, ploty, color='yellow')
# cv2.imwrite('rubric_images/fit_line_{}'.format(filepath.split('/')[-1]), out_img)
plt.imshow(out_img);
We can see that the windows are properly following the lane lines even when they are not that clear or separated by a good amount of distance.
We might need a class that will store the state of whether the margin was detected, some hyperparameters that do not change, etc
In [14]:
class Line():
'''
Store all the parameters related to the left and right lines independently as instance variables
as well as Hyperparameters common to both as class variables.
'''
_nwindows = 10
_margin = 80
_minpix = 40
def __init__(self):
self.detected = False
self.recent_fit = []
self.best_fit = None
self.current_fit = [np.array([False])]
self.diffs = np.array([0,0,0], dtype='float')
self.allx = None
self.ally = None
self.counter = 0
In [15]:
def pipeline(img):
'''
The pipeline involves the below steps
- find the thresholded binary image w.r.t to color spaces
- process the binary image to remove distortions
- warp the image to get a birds eye view i.e. top view perspective
:param RGB frame or image depending on whether the video is getting processed or a single image
:return binary warped image
'''
s_thresh, v_thresh = (200,240), (230,255)
binary_img = threshold_image(img, s_thresh, v_thresh)
undist_img = undistort_image(binary_img)
binary_warped, M, MInv = warp(undist_img)
return binary_warped, M, MInv
def second_ord_poly(line, X):
'''
A formula for second order polynomial.
:param numeric coefficients of the quadratic equation
:param X is the solution for the quadratic equation
:return formula for second order polynomial
'''
A = line[0]
B = line[1]
C = line[2]
return (A*X**2)+(B*X)+C
def search_when_undetected(img):
'''
Called when either it is first time x values are to be found which helps us begin drawing the polynomials or
in the case that the margin has not been found for a set threshold.
:param RGB image
:return None
'''
binary_warped, M, MInv = pipeline(img)
out_img = np.dstack((binary_warped, binary_warped, binary_warped))*255
histogram = np.sum(binary_warped[int(binary_warped.shape[0]/2):,:], axis=0)
midpoint = np.int(histogram.shape[0]/2)
leftx_base = np.argmax(histogram[:midpoint])
rightx_base = np.argmax(histogram[midpoint:]) + midpoint
nwindows = Line._nwindows
margin = Line._margin
minpix = Line._minpix
window_height = np.int(binary_warped.shape[0]/nwindows)
nonzero = binary_warped.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
leftx_current = leftx_base
rightx_current = rightx_base
left_lane_inds = []
right_lane_inds = []
for window in range(nwindows):
win_y_low = binary_warped.shape[0] - (window+1)*window_height
win_y_high = binary_warped.shape[0] - window*window_height
win_xleft_low = leftx_current - margin
win_xleft_high = leftx_current + margin
win_xright_low = rightx_current - margin
win_xright_high = rightx_current + margin
cv2.rectangle(out_img,(win_xleft_low,win_y_low),(win_xleft_high,win_y_high),(0,255,0), 2)
cv2.rectangle(out_img,(win_xright_low,win_y_low),(win_xright_high,win_y_high),(0,255,0), 2)
good_left_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xleft_low) & (nonzerox < win_xleft_high)).nonzero()[0]
good_right_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xright_low) & (nonzerox < win_xright_high)).nonzero()[0]
left_lane_inds.append(good_left_inds)
right_lane_inds.append(good_right_inds)
if len(good_left_inds) > minpix:
leftx_current = np.int(np.mean(nonzerox[good_left_inds]))
if len(good_right_inds) > minpix:
rightx_current = np.int(np.mean(nonzerox[good_right_inds]))
left_lane_inds = np.concatenate(left_lane_inds)
right_lane_inds = np.concatenate(right_lane_inds)
leftx = nonzerox[left_lane_inds]
lefty = nonzeroy[left_lane_inds]
rightx = nonzerox[right_lane_inds]
righty = nonzeroy[right_lane_inds]
try:
n = 5
left_line.current_fit = np.polyfit(lefty, leftx, 2)
left_line.all_x = leftx
left_line.all_y = lefty
left_line.recent_fit.append(left_line.current_fit)
if len(left_line.recent_fit) > 1:
left_line.diffs = (left_line.recent_fit[-2] - left_line.recent_fit[-1]) / left_line.recent_fit[-2]
left_line.recent_fit = left_line.recent_fit[-n:]
left_line.best_fit = np.mean(left_line.recent_fit, axis = 0)
left_fit = left_line.current_fit
left_line.detected = True
left_line.counter = 0
except TypeError:
left_fit = left_line.best_fit
left_line.detected = False
except np.linalg.LinAlgError:
left_fit = left_line.best_fit
left_line.detected = False
try:
n = 5
right_line.current_fit = np.polyfit(righty, rightx, 2)
right_line.all_x = rightx
right_line.all_y = righty
right_line.recent_fit.append(right_line.current_fit)
if len(right_line.recent_fit) > 1:
right_line.diffs = (right_line.recent_fit[-2] - right_line.recent_fit[-1]) / right_line.recent_fit[-2]
right_line.recent_fit = right_line.recent_fit[-n:]
right_line.best_fit = np.mean(right_line.recent_fit, axis = 0)
right_fit = right_line.current_fit
right_line.detected = True
right_line.counter = 0
except TypeError:
right_fit = right_line.best_fit
right_line.detected = False
except np.linalg.LinAlgError:
right_fit = right_line.best_fit
right_line.detected = False
def count_check(line):
'''
As mentioned before, if the threshold is met which means the margins have not been found for some threshold
set of times, the function sets the detected flag to False. For the next image, the x values are counted
from scratch.
:param current line object
:return None
'''
if line.counter >= 5:
line.detected = False
def draw_lane_lines(img):
'''
For efficiency sake, we do not find the x values from scratch each time, if the values had been detected in
last frame, it is very unlikely that there is going to be a lot of change how the next set of values will
change and hence we use the prior know values and focus on fitting the polynomial.
:param RGB image/frame
:return resultant image which is a original image with lanes drawn on it
'''
binary_warped, M, MInv = pipeline(img)
if left_line.detected == False | right_line.detected == False:
search_when_undetected(img)
left_fit = left_line.current_fit
right_fit = right_line.current_fit
nonzero = binary_warped.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
margin = 100
left_lane_inds = ((nonzerox > (left_fit[0]*(nonzeroy**2) + left_fit[1]*nonzeroy + left_fit[2] - margin)) & (nonzerox < (left_fit[0]*(nonzeroy**2) + left_fit[1]*nonzeroy + left_fit[2] + margin)))
right_lane_inds = ((nonzerox > (right_fit[0]*(nonzeroy**2) + right_fit[1]*nonzeroy + right_fit[2] - margin)) & (nonzerox < (right_fit[0]*(nonzeroy**2) + right_fit[1]*nonzeroy + right_fit[2] + margin)))
leftx = nonzerox[left_lane_inds]
lefty = nonzeroy[left_lane_inds]
rightx = nonzerox[right_lane_inds]
righty = nonzeroy[right_lane_inds]
try:
n = 5
left_line.current_fit = np.polyfit(lefty, leftx, 2)
left_line.all_x = leftx
left_line.all_y = lefty
left_line.recent_fit.append(left_line.current_fit)
if len(left_line.recent_fit) > 1:
left_line.diffs = (left_line.recent_fit[-2] - left_line.recent_fit[-1]) / left_line.recent_fit[-2]
left_line.recent_fit = left_line.recent_fit[-n:]
left_line.best_fit = np.mean(left_line.recent_fit, axis = 0)
left_fit = left_line.current_fit
left_line.detected = True
left_line.counter = 0
except TypeError:
left_fit = left_line.best_fit
count_check(left_line)
except np.linalg.LinAlgError:
left_fit = left_line.best_fit
count_check(left_line)
try:
n = 5
right_line.current_fit = np.polyfit(righty, rightx, 2)
right_line.all_x = rightx
right_line.all_y = righty
right_line.recent_fit.append(right_line.current_fit)
if len(right_line.recent_fit) > 1:
right_line.diffs = (right_line.recent_fit[-2] - right_line.recent_fit[-1]) / right_line.recent_fit[-2]
right_line.recent_fit = right_line.recent_fit[-n:]
right_line.best_fit = np.mean(right_line.recent_fit, axis = 0)
right_fit = right_line.current_fit
right_line.detected = True
right_line.counter = 0
except TypeError:
right_fit = right_line.best_fit
count_check(right_line)
except np.linalg.LinAlgError:
right_fit = right_line.best_fit
count_check(right_line)
fity = np.linspace(0, binary_warped.shape[0]-1, binary_warped.shape[0] )
fit_leftx = left_fit[0]*fity**2 + left_fit[1]*fity + left_fit[2]
fit_rightx = right_fit[0]*fity**2 + right_fit[1]*fity + right_fit[2]
out_img = np.dstack((binary_warped, binary_warped, binary_warped))*255
window_img = np.zeros_like(out_img)
out_img[nonzeroy[left_lane_inds], nonzerox[left_lane_inds]] = [255, 0, 0] #red
out_img[nonzeroy[right_lane_inds], nonzerox[right_lane_inds]] = [0, 0, 255] #blue
left_line_window1 = np.array([np.transpose(np.vstack([fit_leftx-margin, fity]))])
left_line_window2 = np.array([np.flipud(np.transpose(np.vstack([fit_leftx+margin, fity])))])
left_line_pts = np.hstack((left_line_window1, left_line_window2))
right_line_window1 = np.array([np.transpose(np.vstack([fit_rightx-margin, fity]))])
right_line_window2 = np.array([np.flipud(np.transpose(np.vstack([fit_rightx+margin, fity])))])
right_line_pts = np.hstack((right_line_window1, right_line_window2))
y_eval = np.max(fity)
left_curverad = ((1 + (2*left_fit[0]*y_eval + left_fit[1])**2)**1.5) / np.absolute(2*left_fit[0])
right_curverad = ((1 + (2*right_fit[0]*y_eval + right_fit[1])**2)**1.5) / np.absolute(2*right_fit[0])
ym_per_pix = 30/720
xm_per_pix = 3.7/700
left_fit_cr = np.polyfit(left_line.all_y*ym_per_pix, left_line.all_x*xm_per_pix, 2)
right_fit_cr = np.polyfit(right_line.all_y*ym_per_pix, right_line.all_x*xm_per_pix, 2)
left_curverad = ((1 + (2*left_fit_cr[0]*y_eval*ym_per_pix + left_fit_cr[1])**2)**1.5) / np.absolute(2*left_fit_cr[0])
right_curverad = ((1 + (2*right_fit_cr[0]*y_eval*ym_per_pix + right_fit_cr[1])**2)**1.5) / np.absolute(2*right_fit_cr[0])
avg_rad = round(np.mean([left_curverad, right_curverad]),0)
rad_text = "Radius of Curvature = {}(m)".format(avg_rad)
middle_of_image = img.shape[1] / 2
car_position = middle_of_image * xm_per_pix
left_line_base = second_ord_poly(left_fit_cr, img.shape[0] * ym_per_pix)
right_line_base = second_ord_poly(right_fit_cr, img.shape[0] * ym_per_pix)
lane_mid = (left_line_base+right_line_base)/2
dist_from_center = lane_mid - car_position
if dist_from_center >= 0:
center_text = "{} meters left of center".format(round(dist_from_center, 2))
else:
center_text = "{} meters right of center".format(round(-dist_from_center, 2))
font = cv2.FONT_HERSHEY_SIMPLEX
cv2.putText(img, center_text, (10,50), font, 1,(255,255,255),2)
cv2.putText(img, rad_text, (10,100), font, 1,(255,255,255),2)
warp_zero = np.zeros_like(binary_warped).astype(np.uint8)
color_warp = np.dstack((warp_zero, warp_zero, warp_zero))
pts_left = np.array([np.transpose(np.vstack([fit_leftx, fity]))])
pts_right = np.array([np.flipud(np.transpose(np.vstack([fit_rightx, fity])))])
pts = np.hstack((pts_left, pts_right))
cv2.fillPoly(color_warp, np.int_([pts]), (0,255, 0))
unwarped = cv2.warpPerspective(color_warp, MInv, (img.shape[1], img.shape[0]))
result = cv2.addWeighted(img, 1, unwarped, 0.3, 0)
return result
def process_image(image):
'''
Function to delegate the entire work of processing the pipeline and drawing the ploymials on the original
image.
:param RGB image
:return resultant image with lanes drawn
'''
return draw_lane_lines(image)
In [16]:
left_line = Line()
right_line = Line()
Radius of Curvature¶
In [17]:
filepath = 'test_images/straight_lines2.jpg'
img = cv2.cvtColor(cv2.imread(filepath), cv2.COLOR_BGR2RGB)
result = process_image(img)
plt.figure(figsize=(15,9))
# cv2.imwrite('rubric_images/lane_drawn_roc_offset_{}'.format(filepath.split('/')[-1]), cv2.cvtColor(result, cv2.COLOR_BGR2RGB))
plt.imshow(result);
Project Video¶
Please check the file named "project_video_output.mp4" which has the video required to pass the project.
In [18]:
vid_input = 'project_video.mp4'
vid_output = 'project_video_output.mp4'
clip1 = VideoFileClip(vid_input)
vid_clip = clip1.fl_image(process_image)
vid_clip.write_videofile(vid_output, audio=False)